Key

Plots:

Summary plots - Sample strategy is on the y-axis and number of sites is on the x-axis. Each plot is paired by parameter level vertically and the values in the cells are the mean value across all of the simulations for that parameter level. Note that each average encompasses all of the other varying simulation parameters.

Full plots - Sample strategy is on the y-axis and number of sites is on the x-axis. Each plot represents a unique simulation and the values in the cells are the mean value across all of the 10 iterations of that simulation across all three unique landscape seeds (i.e., all three sets of Neutral Landscape Models) for a total of 30 replicates. For these plots: K = population size (not to be confused with the number of latent factors (K)), phi = selection strength, m = migration, H = spatial autocorrelation, r = correlation between environmental layers.

Methods:

K - number of latent factors used in LFMM

TPR - True Positive Rate

FDR - False Discovery Rate

lasso vs ridge- “lasso” and “ridge” are different methods utilized by LFMM that have different penalization functions (Caye et al., 2019)

pRDA - partial RDA conditioning on two PC axes to control for population genetic structure

1. LFMM

1.1 Individual sampling

1.1.1 Summary plots

1.1.2 Linear mixed effects models

Only results from the ridge method are used in the final models

TPR

Linear mixed effect model
statistic ~ nsamp + sampstrat + K + m + phi + H + r + (1 | seed)
Predictors Fixed Effects Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
Sample number 0.0002 2.7840 2.7840 1 15.3480K 309.0550 1.6 × 10−68***
Population size 0.0260 2.5944 2.5944 1 15.3480K 288.0033 5.2 × 10−64***
Migration 0.0529 10.7646 10.7646 1 15.3480K 1194.9667 3.2 × 10−252***
Selection strength 0.0921 32.5538 32.5538 1 15.3480K 3613.7662 0.0***
Spatial autocorrelation 0.0950 34.6821 34.6821 1 15.3480K 3850.0347 0.0***
Environmental correlation −0.0166 1.0521 1.0521 1 15.3480K 116.7964 4.0 × 10−27***
*** p < 0.001
Tukey test
pairwise ~ sampstrat
Contrast Estimate SE Z ratio p
EG - G −0.0008 0.0022 −0.3907 0.9797586
EG - R 0.0028 0.0022 1.3075 0.5581221
EG - T*** 0.0143 0.0022 6.5824 2.8 × 10−10***
G - R 0.0037 0.0022 1.6982 0.3245660
G - T*** 0.0151 0.0022 6.9731 1.9 × 10−11***
R - T*** 0.0114 0.0022 5.2749 7.9 × 10−7***
*** p < 0.001

FDR

Linear mixed effect model
statistic ~ nsamp + sampstrat + K + m + phi + H + r + (1 | seed)
Predictors Fixed Effects Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
Sample number −0.0028 604.6185 604.6185 1 15.3480K 4074.163713 0.0***
Population size 0.0091 0.3177 0.3177 1 15.3480K 2.140991 0.140
Migration 0.0535 11.0042 11.0042 1 15.3480K 74.150771 7.9 × 10−18***
Selection strength −0.0341 4.4620 4.4620 1 15.3480K 30.066848 4.2 × 10−8***
Spatial autocorrelation 0.0813 25.3925 25.3925 1 15.3480K 171.105060 6.9 × 10−39***
Environmental correlation −0.0109 0.4526 0.4526 1 15.3480K 3.049905 0.081
*** p < 0.001
Tukey test
pairwise ~ sampstrat
Contrast Estimate SE Z ratio p
EG - G 0.0097 0.0088 1.1006 0.68917026
EG - R −0.0193 0.0088 −2.1984 0.12363814
EG - T*** −0.0435 0.0088 −4.9431 4.6 × 10−6***
G - R** −0.0290 0.0088 −3.2989 5.4 × 10−3**
G - T*** −0.0531 0.0088 −6.0437 9.0 × 10−9***
R - T* −0.0241 0.0088 −2.7448 0.03081891*
*** p < 0.001
** p < 0.01
* p < 0.05

1.1.3 Full plots

K

TPR

FDR

Total number of loci

1.2 Site sampling

1.2.1 Summary plots

1.1.2 Linear mixed effects models

Only results from the ridge method are used in the final models

TPR

Linear mixed effect model
statistic ~ nsamp + sampstrat + K + m + phi + H + r + (1 | seed)
Predictors Fixed Effects Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
Sample number −0.0010 1.4682 1.4682 1 34.5490K 112.1134 3.7 × 10−26***
Population size 0.0223 4.2973 4.2973 1 34.5490K 328.1426 5.3 × 10−73***
Migration 0.0375 12.1734 12.1734 1 34.5490K 929.5717 1.7 × 10−201***
Selection strength 0.0757 49.5136 49.5136 1 34.5490K 3780.8896 0.0***
Spatial autocorrelation 0.1029 91.4338 91.4338 1 34.5490K 6981.9351 0.0***
Environmental correlation −0.0242 5.0647 5.0647 1 34.5490K 386.7470 1.2 × 10−85***
*** p < 0.001
Tukey test
pairwise ~ sampstrat
Contrast Estimate SE Z ratio p
EG - EQ*** 0.0098 0.0015 6.4694 3.0 × 10−10***
EG - R*** 0.0093 0.0015 6.1743 2.0 × 10−9***
EQ - R −0.0004 0.0015 −0.2950 0.9531481
*** p < 0.001

FDR

Linear mixed effect model
statistic ~ nsamp + sampstrat + K + m + phi + H + r + (1 | seed)
Predictors Fixed Effects Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
Sample number 0.0090 119.4872 119.4872 1 34.5510K 1.371798e+03 1.7 × 10−294***
Population size 0.0007 0.0043 0.0043 1 34.5510K 4.942033e-02 0.820
Migration −0.0010 0.0095 0.0095 1 34.5510K 1.086153e-01 0.740
Selection strength −0.0215 4.0100 4.0100 1 34.5510K 4.603750e+01 1.2 × 10−11***
Spatial autocorrelation −0.0063 0.3439 0.3439 1 34.5510K 3.948046e+00 0.047**
Environmental correlation 0.0029 0.0707 0.0707 1 34.5510K 8.115947e-01 0.370
*** p < 0.001
** p < 0.05
Tukey test
pairwise ~ sampstrat
Contrast Estimate SE Z ratio p
EG - EQ*** 0.0124 0.0039 3.1771 4.2 × 10−3***
EG - R −0.0008 0.0039 −0.1998 0.9782368
EQ - R*** −0.0131 0.0039 −3.3769 2.1 × 10−3***
*** p < 0.01

1.2.3 Full plots

K

TPR

FDR

Total number of loci

1.3 Latent factor test

To determine the effect of K-selection on performance we ran LFMM using a constant K for all of the subsampled datasets from the same simulation (i.e., all sample sizes and strategies had the same K) and compared that to K-selection based on each sub-sampled dataset (i.e., K was allowed to vary by sample size and strategy). The constant K for each simulation was selected using a dataset of 1000 randomly selected individuals. For this test, we only evaluated the “ridge” and not the “lasso” method as our earlier tests demonstrated that, in general, the “ridge” method performed better. We compared the results using the same statistics (i.e., TPR and FDR) and linear mixed effects models. Overall, the final results did not vary substantially between constant and variable K and the effects of all the parameters tested remained the same.

Individual sampling

TPR

Linear mixed effect model
statistic ~ nsamp + sampstrat + K + m + phi + H + r + (1 | seed)
Predictors Fixed Effects Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
Sample number 0.0002 2.5777 2.5777 1 15.3480K 285.6968 1.6 × 10−63***
Population size 0.0267 2.7327 2.7327 1 15.3480K 302.8711 3.4 × 10−67***
Migration 0.0530 10.7778 10.7778 1 15.3480K 1194.5434 3.9 × 10−252***
Selection strength 0.0931 33.2480 33.2480 1 15.3480K 3684.9903 0.0***
Spatial autocorrelation 0.0961 35.4706 35.4706 1 15.3480K 3931.3324 0.0***
Environmental correlation −0.0166 1.0521 1.0521 1 15.3480K 116.6116 4.4 × 10−27***
*** p < 0.001
Tukey test
pairwise ~ sampstrat
Contrast Estimate SE Z ratio p
EG - G −0.0013 0.0022 −0.6157 0.9271278
EG - R 0.0025 0.0022 1.1563 0.6545013
EG - T*** 0.0140 0.0022 6.4420 7.1 × 10−10***
G - R 0.0038 0.0022 1.7719 0.2869064
G - T*** 0.0153 0.0022 7.0577 1.0 × 10−11***
R - T*** 0.0115 0.0022 5.2858 7.5 × 10−7***
*** p < 0.001

FDR

Linear mixed effect model
statistic ~ nsamp + sampstrat + K + m + phi + H + r + (1 | seed)
Predictors Fixed Effects Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
Sample number −0.0029 646.8019 646.8019 1 15.3480K 4374.882787 0.0***
Population size 0.0140 0.7517 0.7517 1 15.3480K 5.084648 0.024**
Migration 0.0547 11.4783 11.4783 1 15.3480K 77.637772 1.4 × 10−18***
Selection strength −0.0357 4.8980 4.8980 1 15.3480K 33.129151 8.8 × 10−9***
Spatial autocorrelation 0.0770 22.7504 22.7504 1 15.3480K 153.880959 3.6 × 10−35***
Environmental correlation −0.0088 0.2972 0.2972 1 15.3480K 2.009990 0.160
*** p < 0.001
** p < 0.05
Tukey test
pairwise ~ sampstrat
Contrast Estimate SE Z ratio p
EG - G 0.0068 0.0088 0.7754 0.86563528
EG - R −0.0184 0.0088 −2.0991 0.15333943
EG - T*** −0.0424 0.0088 −4.8354 7.9 × 10−6***
G - R** −0.0252 0.0088 −2.8745 0.02109283**
G - T*** −0.0492 0.0088 −5.6108 1.2 × 10−7***
R - T** −0.0240 0.0088 −2.7363 0.03156906**
*** p < 0.001
** p < 0.05

Site sampling

TPR

Linear mixed effect model
statistic ~ nsamp + sampstrat + K + m + phi + H + r + (1 | seed)
Predictors Fixed Effects Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
Sample number −0.0040 5.8037 5.8037 1 8.6290K 552.7327 1.6 × 10−118***
Population size 0.0190 0.7830 0.7830 1 8.6290K 74.5706 6.9 × 10−18***
Migration 0.0457 4.5032 4.5032 1 8.6290K 428.8741 5.1 × 10−93***
Selection strength 0.0731 11.5391 11.5391 1 8.6290K 1098.9616 6.1 × 10−227***
Spatial autocorrelation 0.1048 23.7248 23.7248 1 8.6290K 2259.5020 0.0***
Environmental correlation −0.0252 1.3688 1.3688 1 8.6290K 130.3630 5.6 × 10−30***
*** p < 0.001
Tukey test
pairwise ~ sampstrat
Contrast Estimate SE Z ratio p
EG - EQ 0.0040 0.0027 1.4948 0.29338489
EG - R*** 0.0109 0.0027 4.0505 1.5 × 10−4***
EQ - R** 0.0069 0.0027 2.5556 0.02857052**
*** p < 0.001
** p < 0.05

FDR

Linear mixed effect model
statistic ~ nsamp + sampstrat + K + m + phi + H + r + (1 | seed)
Predictors Fixed Effects Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
Sample number −0.0073 19.7739 19.7739 1 8.6310K 836.14415238 1.4 × 10−175***
Population size 0.0017 0.0065 0.0065 1 8.6310K 0.27476374 0.60
Migration 0.0006 0.0007 0.0007 1 8.6310K 0.02904367 0.86
Selection strength −0.0292 1.8461 1.8461 1 8.6310K 78.06102539 1.2 × 10−18***
Spatial autocorrelation 0.0009 0.0016 0.0016 1 8.6310K 0.06739987 0.80
Environmental correlation 0.0018 0.0071 0.0071 1 8.6310K 0.29928678 0.58
*** p < 0.001
Tukey test
pairwise ~ sampstrat
Contrast Estimate SE Z ratio p
EG - EQ 0.0006 0.0041 0.1489 0.9878524
EG - R −0.0017 0.0041 −0.4238 0.9057716
EQ - R −0.0023 0.0041 −0.5727 0.8347053

Comparison of effects

Sampling Population Genetics Landscape
Sampling type Sample number Population size Migration Selection strength Spatial autocorrelation Environmental correlation
TPR
Constant K individual 0.0002 0.0267 0.0530 0.0931 0.0961 −0.0166
Variable K individual 0.0002 0.0260 0.0529 0.0921 0.0950 −0.0166
Constant K site −0.0040 0.0190 0.0457 0.0731 0.1048 −0.0252
Variable K site −0.0010 0.0223 0.0375 0.0757 0.1029 −0.0242
FDR
Constant K individual −0.0029 0.0140 0.0547 −0.0357 0.0770 n.s.
Variable K individual −0.0028 n.s. 0.0535 −0.0341 0.0813 n.s.
Constant K site −0.0073 n.s. n.s. −0.0292 n.s. n.s.
Variable K site 0.0090 n.s. n.s. −0.0215 −0.0063 n.s.

2. RDA

2.1 Individual sampling

2.1.1 Summary plots

2.1.2 Linear mixed effects models

Only results from the standard RDA (not the partial RDA) are used in the final models

TPR

Linear mixed effect model
statistic ~ nsamp + sampstrat + K + m + phi + H + r + (1 | seed)
Predictors Fixed Effects Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
Sample number 0.0002 2.8394 2.8394 1 15.3480K 545.389450 1.5 × 10−118***
Population size 0.0165 1.0439 1.0439 1 15.3480K 200.509665 3.1 × 10−45***
Migration 0.0438 7.3555 7.3555 1 15.3480K 1412.856406 7.2 × 10−296***
Selection strength 0.0440 7.4213 7.4213 1 15.3480K 1425.494641 2.2 × 10−298***
Spatial autocorrelation 0.0436 7.2900 7.2900 1 15.3480K 1400.274445 2.3 × 10−293***
Environmental correlation 0.0036 0.0488 0.0488 1 15.3480K 9.371404 2.2 × 10−3**
*** p < 0.001
** p < 0.01
Tukey test
pairwise ~ sampstrat
Contrast Estimate SE Z ratio p
EG - G 0.0005 0.0016 0.2768 0.9926010
EG - R 0.0029 0.0016 1.7594 0.2931233
EG - T*** 0.0110 0.0016 6.6817 1.4 × 10−10***
G - R 0.0024 0.0016 1.4826 0.4480091
G - T*** 0.0105 0.0016 6.4050 9.0 × 10−10***
R - T*** 0.0081 0.0016 4.9224 5.1 × 10−6***
*** p < 0.001

FDR

Linear mixed effect model
statistic ~ nsamp + sampstrat + K + m + phi + H + r + (1 | seed)
Predictors Fixed Effects Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
Sample number 0.0003 5.7961 5.7961 1 15.3480K 373.26868 3.4 × 10−82***
Population size 0.0153 0.9004 0.9004 1 15.3480K 57.98778 2.8 × 10−14***
Migration 0.0753 21.8007 21.8007 1 15.3480K 1403.96987 4.2 × 10−294***
Selection strength 0.0647 16.0600 16.0600 1 15.3480K 1034.26485 1.2 × 10−219***
Spatial autocorrelation 0.0610 14.2799 14.2799 1 15.3480K 919.62815 3.1 × 10−196***
Environmental correlation 0.0091 0.3176 0.3176 1 15.3480K 20.45101 6.2 × 10−6***
*** p < 0.001
Tukey test
pairwise ~ sampstrat
Contrast Estimate SE Z ratio p
EG - G −0.0010 0.0028 −0.3555 0.98460216
EG - R 0.0020 0.0028 0.6959 0.89870855
EG - T*** 0.0085 0.0028 2.9888 0.01487071***
G - R 0.0030 0.0028 1.0514 0.71907362
G - T** 0.0095 0.0028 3.3443 4.6 × 10−3**
R - T 0.0065 0.0028 2.2929 0.09963697
*** p < 0.05
** p < 0.01

1.1.3 Full plots

TPR

FDR

Total number of loci

2.2 Site sampling

2.2.1 Summary plots

2.2.2 Linear mixed effects models

Only results from the standard RDA (not the partial RDA) are used in the final models

TPR

Linear mixed effect model
statistic ~ nsamp + sampstrat + K + m + phi + H + r + (1 | seed)
Predictors Fixed Effects Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
Sample number 0.0015 0.8335 0.8335 1 8.6290K 232.154396 9.5 × 10−52***
Population size 0.0173 0.6489 0.6489 1 8.6290K 180.732574 8.6 × 10−41***
Migration 0.0322 2.2403 2.2403 1 8.6290K 623.983518 5.0 × 10−133***
Selection strength 0.0321 2.2322 2.2322 1 8.6290K 621.743005 1.4 × 10−132***
Spatial autocorrelation 0.0315 2.1447 2.1447 1 8.6290K 597.363316 1.3 × 10−127***
Environmental correlation 0.0019 0.0081 0.0081 1 8.6290K 2.261166 0.13
*** p < 0.001
Tukey test
pairwise ~ sampstrat
Contrast Estimate SE Z ratio p
EG - EQ −0.0001 0.0016 −0.0550 0.9983351
EG - R*** 0.0076 0.0016 4.8378 3.9 × 10−6***
EQ - R*** 0.0077 0.0016 4.8928 3.0 × 10−6***
*** p < 0.001

FDR

Linear mixed effect model
statistic ~ nsamp + sampstrat + K + m + phi + H + r + (1 | seed)
Predictors Fixed Effects Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
Sample number 0.0018 1.1681 1.1681 1 8.6290K 140.00975 4.7 × 10−32***
Population size 0.0133 0.3801 0.3801 1 8.6290K 45.55864 1.6 × 10−11***
Migration 0.0378 3.0791 3.0791 1 8.6290K 369.07182 1.4 × 10−80***
Selection strength 0.0336 2.4370 2.4370 1 8.6290K 292.10683 2.0 × 10−64***
Spatial autocorrelation 0.0297 1.8994 1.8994 1 8.6290K 227.67214 8.5 × 10−51***
Environmental correlation 0.0144 0.4449 0.4449 1 8.6290K 53.32587 3.1 × 10−13***
*** p < 0.001
Tukey test
pairwise ~ sampstrat
Contrast Estimate SE Z ratio p
EG - EQ*** 0.0057 0.0024 2.3728 0.04642683***
EG - R** 0.0085 0.0024 3.5382 1.2 × 10−3**
EQ - R 0.0028 0.0024 1.1654 0.47395300
*** p < 0.05
** p < 0.01

2.2.3 Full plots

TPR

FDR

Total number of loci